Originally, computers were programmed in binary code. The programmer had to program the bits (1s and 0s) individually. This was not a very efficient system, and soon led to the development of assembly language. Assembly language allowed programmers to use instructions that represented specific sequences of bits; when executed in sequence, the assembly language instructions constituted a program.
One feature of assembly language programming is the ability to jump from one location in the program to another. This functionality allows for the implementation of branching code, such as the “if-then” and “if-then-else” statements of higher level programming languages, where code was executed only if certain conditions were met. This functionality also enables the execution of subroutines—portions of code that can be invoked repeatedly, and from different places within the program.
Most computers implement the call of a subroutine using a concept call a stack. A stack is a portion of memory which is accessed on a “last in, first out” basis. When a transfer of processing to a subroutine is to be performed, the computer “pushes”, or places, certain information on the stack. This information includes, among other data, some memory needed for variables used in the subroutine and the address to which control is to be returned when the subroutine is finished.
As a design for managing memory used in program execution in the computer, this model works quite well. But many modem operating systems are programmed in a particular higher level programming language: C. One of the features of C is that C does not check whether writes to memory are compliant with how the memory is allocated. For example, C does not verify whether the data being written to memory is of the type for which the memory space was allocated, or that the data being written will fit into the allocated memory space. Some programmers consider this a strength of the language, in that the program does not spend time enforcing such conditions. If the programmer considers type-checking or buffer overflow to be important, the programmer can test for such conditions in the program; otherwise, the programmer can trust that the program will not be misused.
But this strength is also a potential weakness. For example, Microsoft® releases many patches to its Windows® family of operating systems specifically to patch security flaws relating to buffer overflow issues. (Microsoft and Windows are registered trademarks of Microsoft Corporation in the United States and other countries.) The problem is that if more data is written than the target memory address can hold, the excess ends up overwriting other values in memory allocated for other purposes.
Hackers often use buffer overflow vulnerabilities to try and take control of user's computers. Specifically, the hackers attempt to overflow a buffer by writing a particular piece of data into the buffer. The hackers design the data so that it includes the code they use to take over the computer, along with an address where their code will be located in memory. If the hackers can overflow the buffer so that the address where their code is located overwrites the return address on the stack, then the hacker's code will be executed when the subroutine completes. The hacker can then gain control of the computer.
A need remains for a way to addresses these and other problems associated with the prior art.